Skip to content

1 1 restore feature branch #4349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from
Draft

Conversation

karol-kokoszka
Copy link
Collaborator

DRAFT PR
(for CI availability)


Please make sure that:

  • Code is split to commits that address a single change
  • Commit messages are informative
  • Commit titles have module prefix
  • Commit titles have issue nr. suffix

VAveryanov8 and others added 9 commits April 14, 2025 16:45
This adds a new task type for fast vnode restore.

Relates: #4200
* feat(1-1-restore): adds new 1-1-restore sub command

This introduces new cli sub command for the 1-1-restore

Relates: #4200
…equal (#4230)

feat(1-1-restore): validates if source and target clusters nodes are equal

This adds a validation stage for 1-1-restore. The logic is as follows:
- Collect node information for the source cluster from backup manifests
- Collect node information for the target cluster from the Scylla API
- Compare each source node with its corresponding target node

Apart from that nodes-mapping param is validated to ensure that each node has exactly one mapping.

Refs: #4201
feat(1-1-restore): adds copy data stage + loadSStables call

This adds copy data stage for the 1-1-restore with simple integration
test. Data is copied by each node, table by table and after each table
loadSStables is called.
Note: integration tests with TABLETS=enabled will be skipped as 1-1-restore should only
work with v-nodes.

Refs:  #4202 #4203
This adds support for restoring views in 1-1-restore.
Views should be dropped before copy data stage and then re-created to
avoid data inconsistency between base table and view.

Refs: #4252
…l filtering only (#4256)

This reduces functionality of `--keyspace` parameter to support only:
 - keyspace level filtering. Table level filtering (system.table_name) is forbidden.
 - only include filtering. Excluding(!) is forbidden.
 - wildcards (*) are forbidden. The only exception is when the only filter that is provided is a single character (*),
meaning that all keyspaces should be included into restore process.
This is done by allowing to filter on keyspaces that are already exists in the cluster.

Refs: #4253
feat(1-1-restore): sets tombstone_gc mode to repair

This adds skipping of local system tables - the same way we have in
regular restore service.

This sets tombstone_gc mode to 'repair' to avoid data resurrection.
If tables mode is already 'repair', 'disabled' or 'immediate', then it won't be changed.

Refs: #4258
This adds progress tracking of various 1-1-restore stage.
Each progress update represented by RunProgress model saved into db.
w.getProgress is used to aggregate current task progress by Keyspaces,
Tables and Hosts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants